I'm not entirely sure that I can just yet. :-P

On my H2O I found I wasn't able to scroll, even though it's working on the emulator.

Confirmed: scrolling with swipe does not work in scrollhtmlwidget (I scroll with tap mostly). It does not scroll for me either in the emulator.
Small typo (a natural/visual half-conscious corretion I guess). Easy to fix:

     if Device:isTouchDevice() then
         self.ges_events = {
-            SwipeScrollText = {
+            ScrollText = {
                 GestureRange:new{

Should I add it to this PR ? or you prefer an individual one?

I suppose a separate quickie is better for history.

Another crash that I figure should theoretically be avoided is the error throwing, i.e., sticking it behind an if dbg.is_on then and doing something a little more user-friendly otherwise.

But that's more of a hypothetical thing. Since I'm not completely sure what the best friendly thing to do is, other than not closing the program[1] I won't throw in a quickie for that one and I won't let it hold up anything tomorrow. ;-)

[1] Retry yet again with known good HTML saying something about an error? Close the widget and show some kind of InfoMessage instead? Leave it white with an InfoMessage? Some other option?

I don't really know.
But I assume that with the html to plain text conversion just done before, there's no real reason for it to fail (except for other things like not enough memory, etc...), so we should be fine.

Sharing the html fix funcs I made for the 3 HTML french dicts I have (to be put alongside the .ifo and renamed as it with a .lua extension instead of .ifo):
PetitRobert2007_1.1.lua.txt
PetitRobert2003_v4.lua.txt
XMLittre.lua.txt

We should look into a sensible way of including the XML Littré fix. (What's the problem with something that explicitly has XML in the name? Some custom XML DTD rather than HTML?)

Yes, some XML which is not fully HTML. For example, in my .lua:

    html = html:gsub('<big>', '<span style="font-weight: bold; font-size: 1.1em">')
    html = html:gsub(' foreground="', ' style="color: ')

(+ some other cosmetics changes for my taste, like replacing colors with smaller italic fonts, etc...)
(But I'm not not sure I didnt explicitely changed sametypesequence=x to sametypesequence=h - so it may by default displays as text.)
(Or may be it was bad HTML, and the conversion to simple HTML is OK enough to have it looks as text like it did before - and I must have by chance see it was indeed some HTML, and tried to make something better out of it.)

Hm, how old is that dictionary? It's being continuously corrected, although it looks like it remains an XML version of ye olde Littré, see here.

Is there a script somewhere that automates the conversion from the XML Littré source to stardict? There's https://bitbucket.org/Mytskine/xmlittre-web but I'm not seeing any XSLT in that repo.

https://www.littre.org/faq

Anyway, just something that one could consider as a lunch or breakfast project. :-P

Oh, didn't know that. Files in mine are dated from 2006 :)

Those curious could investigate whether https://github.com/leafo/web_sanitize could be of any use. Although its purpose is to sanitize (i.e., from scripts and styles), it seems to handle unclosed inner tags. It's a fair bit smaller than libtidy. :-P

It depends on LPeg, but we already include that for lua-Spore.

Note that MuPDF support for xhtml is limited.
For example it doesn't display some items nested in <table>, even though validated by tidy.

I write a lua script to [test if it is possible to] correctly render persian/arabic strings in definitions, but per my tests koreader even doesn't load my script. May you help me please?
persian.lua.txt

MuPDF should already do that?

If you're talking about the KOReader GUI elements they don't, but I'd probably look into leveraging HarfBuzz. Last year we didn't include it, but now we do.

https://github.com/luapower/harfbuzz
https://github.com/luapower/harfbuzz/blob/a4fa5884950166248b78d768eb806bb413ef4cdf/harfbuzz_demo.lua#L162-L163

(Also: https://github.com/ufyTeX/luaharfbuzz)

per my tests koreader even doesn't load my script

You didn't say what you're trying to load and use it.

MuPDF should already do that?

No it doesn't. Letters are seperated and left to right.
Currently I'm working on fixing dictionary results. But regarding rendering books, harfbuzz and mupdf are painfully slow and generated results aren't useful. We may talk about it in a related issue topic.
I'll post more details tomorrow.

No it doesn't.

What exactly is it that it doesn't do in what scenario? See, e.g., #1426 (comment)

I found the reason, the script is run only when the dictionary .ifo has sametypesequence=h . But unforunately MuPDF displays persian characters as ▯, while without MuPDF at least characters are displayed.

What exactly is it that it doesn't do in what scenario?

Here are the screenshots:
Plain text dictionary:

HTML dictionary:

That could be an issue somewhere in how we build or use MuPDF. I'm reasonably sure it's supposed to fall back to Noto.

I even tried to set the font explicitly through css but it seems that MuPDF doesn't respect the font-family attribute at all. (verified on PC)

Setting it where and how? If you're testing MuPDF I would stick to the basics (e.g., a simple XHTML file) without complicating matters through the dictionary. ;-)

If you want to test the dictionary, see, e.g.:

Which makes it really weird that we're seeing undefined characters; maybe it's actually some issue in the sdcv output or how it's passed on?

It definitely listens to the font there, because otherwise you'd be seeing some serif typeface instead of Noto Sans. :-)

My guess is that output is plain garbage, and not MuPDF failing to display Farsi ;).

(i.e., I'd double-check the sanity of the dictionary, and try the sdcv CLI, as @Frenzie suggested).

tried a css with
@page { margin: 0; font-family: freeserif; }

Now farsi characters are displayed as free space (or nothing!). I'm sure that characters are there, because when I touch and hold on them koreader will seek for their definition and displays them there.